Spatio-temporal image reconstruction using sparse regression and secondary information

ABSTRACT

A method of generating an image includes the step of obtaining captured data characterizing an object. The method also includes the step of reconstructing a spatio-temporal image of the object based on the captured data, the spatio-temporal image comprising a plurality of spatial images in respective time intervals, with at least a given one of the spatial images in one of the time intervals being reconstructed using captured data from a frame associated with that time interval and captured data associated with one or more additional frames associated with other time intervals. The method further includes the step of outputting the spatio-temporal image. The obtaining, reconstructing and outputting steps are performed by a processing device comprising a processor coupled to a memory.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a Continuation of U.S. patent application Ser. No.12/790,199, filed on May 28, 2010, the disclosure of which isincorporated herein by reference.

FIELD

The field relates generally to image processing, and more particularlyto techniques for reconstruction of spatio-temporal images from captureddata.

BACKGROUND

Many imaging applications involve generating a sequence oftwo-dimensional (2D) or three-dimensional (3D) images representing aparticular object of interest in respective time intervals. Examplesinclude a wide variety of well-known medical imaging applications suchas magnetic-resonance imaging (MRI), positron emission tomography (PET)and computed tomography (CT). The image sequences generated in these andother imaging applications are also commonly referred to asspatio-temporal images, in that the image data varies both in thespatial domain, that is, within each of the individual 2D or 3D images,as well as in the temporal domain, that is, from time interval to timeinterval.

In a typical arrangement, raw data is captured by a scanner as a seriesof frames, with each frame corresponding to a time interval. Thecaptured data for each frame is processed to reconstruct a spatial imagerepresenting the state of the object in the corresponding time interval.The image reconstruction process may be formulated as a mathematicaloptimization problem based on a physical model characterizing datacapture by the scanner. Spatial images are usually reconstructedindividually on a frame-by-frame by-frame basis, and these spatialimages are then aggregated together to provide the spatio-temporalimage.

In situations involving imaging of repetitive phenomena such as heartbeat or respiration, raw data is often captured for multiple cycles ofrepetition, using so-called “gated” arrangements. Each cycle ofrepetition is subdivided into multiple frames, with each frame of agiven cycle representing a corresponding time interval in that cycle.Spatial images are reconstructed independently for each time intervalusing the frames associated with that time interval, which would includeone frame from each of the multiple cycles. Again, these spatial imagesare then aggregated together to provide the spatio-temporal image.

There is an inherent tradeoff between spatial resolution and temporalresolution in arrangements such as those described above. For example,one can attempt to improve the temporal resolution by increasing thenumber of frames captured within a given time period, but this willdecrease the amount of data captured in each frame, leading to a poorerspatial resolution. Also, it is often desirable to limit the scan timeas certain of the medical imaging applications may involve exposing apatient to significant levels of potentially harmful radiation.

SUMMARY

Illustrative embodiments of the present invention provide improvedtechniques for spatio-temporal image reconstruction in which a givenspatial image associated with one time interval is reconstructed usingcaptured data not only from the corresponding frame but also captureddata from other frames associated with other time intervals. Thesetechniques take both spatial redundancy and temporal redundancy intoaccount in reconstructing a spatio-temporal image.

In addition to exploiting both spatial and temporal redundancy, theimage reconstruction may use information from secondary sources, such asprior specific or general information about the object underinvestigation, to provide further improvements in the quality ofreconstruction.

In accordance with one aspect of the invention, a method of generatingan image includes the step of obtaining captured data characterizing anobject. The method also includes the step of reconstructing aspatio-temporal image of the object based on the captured data, thespatio-temporal image comprising a plurality of spatial images inrespective time intervals, with at least a given one of the spatialimages in one of the time intervals being reconstructed using captureddata from a frame associated with that time interval and captured dataassociated with one or more additional frames associated with other timeintervals. The method further includes the step of outputting thespatio-temporal image. The obtaining, reconstructing and outputtingsteps are performed by a processing device comprising a processorcoupled to a memory.

In accordance with another aspect of the invention, an apparatuscomprises an image reconstruction unit, the image reconstruction unitcomprising a processor. The image reconstruction unit is operative undercontrol of the processor to obtain captured data characterizing anobject, and to reconstruct a spatio-temporal image of the object basedon the captured data. The spatio-temporal image comprises a plurality ofspatial images in respective time intervals and at least a given one ofthe spatial images in one of the time intervals is reconstructed usingcaptured data from a frame associated with that time interval andcaptured data associated with one or more additional frames associatedwith other time intervals.

The illustrative embodiments advantageously allow temporal resolution tobe improved without significantly compromising the spatial resolution,since the captured data from multiple frames is used to reconstruct thespatial image corresponding to a given one of the frames. Also, spatialresolution can be improved without compromising temporal resolution andwithout increasing the scan time. Additionally or alternatively, scantime can be substantially reduced for a given level of spatialresolution, thereby reducing patient exposure to potentially harmfulradiation.

These and other features and advantages of the present invention willbecome more apparent from the accompanying drawings and the followingdetailed description.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an image processing system in anillustrative embodiment of the invention.

FIG. 2 is a block diagram showing a more detailed view of an imagereconstruction unit of the FIG. 1 image processing system as well asadditional elements of the FIG. 1 system.

FIG. 3 is a flow diagram of an image reconstruction process implementedin the image reconstruction module of FIG. 2.

DETAILED DESCRIPTION

The present invention will be illustrated herein in conjunction withexemplary image processing systems and associated techniques forspatio-temporal image reconstruction. It should be understood, however,that the invention is not limited to use with the particular types ofsystems and techniques disclosed. The invention can be implemented in awide variety of other image processing systems, using alternative imagereconstruction techniques.

FIG. 1 shows an image processing system 100 in an illustrativeembodiment of the invention. Data characterizing an object underinvestigation 102 is captured by a data capture unit 104, which maycomprise a scanner, or more particularly a medical imaging scanner suchas an MRI, PET or CT scanner. This raw data is assumed for purposes ofthe present embodiment to be captured as a series of frames, with eachframe corresponding to a time interval. The captured data is processedin an image reconstruction unit 106 to generate a spatio-temporal image.The object 102 may comprise a human subject, an animal subject or anyother object, or a portion of such an object. Other types of scannersmay be used in implementing an embodiment of the invention, includingthose commonly associated with scanning techniques such as functionalMRI (fMRI), nuclear MRI (NMRI), single photon emission computedtomography (SPECT), or high-resolution research tomography (HRRT).

In the conventional image reconstruction techniques describedpreviously, spatial images are usually reconstructed individually on aframe-by-frame basis, and these spatial images are then aggregatedtogether to provide the spatio-temporal image. These conventionaltechniques therefore reconstruct each spatial image using only thecaptured data from its corresponding frame, or corresponding set offrames associated with a common time interval and including one framefrom each cycle in a so-called “gated” arrangement.

The image reconstruction unit 106, in contrast to these conventionalarrangements, is configured such that a given spatial image associatedwith one time interval is reconstructed using captured data not onlyfrom the corresponding frame but also captured data from other framesassociated with other time intervals. Accordingly, the imagereconstruction unit takes both spatial and temporal redundancy intoaccount in reconstructing a spatio-temporal image. The captured datafrom multiple frames is therefore used to reconstruct the spatial imagecorresponding to a given one of the frames. In addition to exploitingboth spatial and temporal redundancy, the image reconstruction unit 106uses information from secondary sources, such as prior specific orgeneral information about the object under investigation, to providefurther improvements in the quality of reconstruction. The secondaryinformation may be information obtained from secondary sources usingother types of imaging modalities or measurements.

Advantageously, this allows temporal resolution to be improved withoutsignificantly compromising the spatial resolution, resulting in improvedspatio-temporal image quality and thereby enhanced diagnosticcapabilities. In addition, spatial resolution can be improved withoutcompromising temporal resolution and without increasing the scan time.The disclosed techniques can also be used to reduce scan time, therebyimproving the efficiency of the scanning equipment (e.g., MRI, PET or CTscanners can take more images in a given time) and reducing the harmfuleffects of scanning on the object being imaged (e.g., by decreasing theradiation exposure to patients being scanned).

As shown in greater detail in FIG. 2, the image reconstruction unit 106comprises a processor 200 coupled to a memory 202 and to interfacecircuitry 204. Associated with the processor 200 are a sparse regressionmodule 210 and a secondary information module 212. These modules areused in spatio-temporal image reconstruction in the illustrativeembodiments, in a manner to be described below.

The memory 202 may store instructions or other program code that isexecuted by the processor 200 in performing functions associated withspatio-temporal image reconstruction. The memory 202 is an example ofwhat is more generally referred to herein as a computer program producthaving embodied therein executable program code, and may compriseelectronic memory such as RAM or ROM, magnetic memory, disk-basedmemory, optical memory or other types of storage elements, in anycombination. The processor 200 may comprise one or more microprocessors,microcontrollers, application-specific integrated circuits (ASICs),graphical processing units (GPUs) or other processing devices, in anycombination, for executing program code stored in memory 202. Themodules 210 and 212 may be implemented at least in part in the form ofsuch program code.

The interface circuitry 204 interfaces the image reconstruction unit 106with a network 220 that is coupled to an external storage unit 222 and adisplay unit 224. The storage unit 222 may be used to store raw datacaptured by the data capture unit 104, processed or unprocessedinformation obtained from secondary sources, as well as spatio-temporalimages generated by the image reconstruction unit 106. The display unit224, which may comprise one or more monitors or projectors, is used forviewing of the spatio-temporal images.

It is to be appreciated that the particular arrangements of system 100and image reconstruction unit 106 as shown in FIGS. 1 and 2 are by wayof illustrative example only. Alternative embodiments may include otherarrangements of system elements suitably configured to supportspatio-temporal image reconstruction of the type described herein.Examples of such arrangements may be found in U.S. patent applicationSer. No. 12/549,964, filed Aug. 28, 2009 and entitled “Reconstruction ofImages Using Sparse Representation,” which is commonly assigned herewithand incorporated by reference herein.

The image reconstruction unit 106 or portions thereof may be implementedat least in part in the form of an integrated circuit. For example, in agiven implementation the image reconstruction unit 106 may be embodiedin a single ASIC or other type of processing device, such as, forexample, GPUs, computers, servers, mobile communication devices, etc.

The spatio-temporal image reconstruction process implemented in theimage reconstruction unit 106 will now be described in greater detail.

The image of the object 102 for a given time interval t may berepresented as an n-dimensional vector x^(t), and the data capture unit104 may be characterized as producing a corresponding m-dimensionalvector of observations given by Y^(t)=Px^(t)+Ψ^(t), where P is an n by mmatrix representing the projection of the object 102 in the data captureunit, and is also referred to as a system matrix, and Ψ^(t) representsthe noise in the measurement process implemented by the data captureunit. The projection matrix will generally depend on the type of datacapture device used, and may be a line integral matrix, a Fouriertransform matrix, a specialized system matrix, etc. Numerous suchprojection matrices are known in the art.

The image reconstruction unit 106 receives Y^(t) from the data captureunit 104 and reconstructs the image vector x^(t) for a given value of tutilizing the observations Y^(t) and the noise Ψ^(t) for multiple valuesof t. LetY=Col(Y ¹ , Y ² , . . . , Y ^(d))represent a vector of size md obtained by stacking the observationsY^(t) for t in the range 1 to d. Similarly letx=Col(x ¹ , x ² , . . . , x ^(d)) andΨ=Col(Ψ¹, Ψ², . . . , Ψ³)respectively represent vectors of size nd and md obtained by stackingthe images x^(t) and the noise Ψ^(t) for t in the range 1 to d. Thevector x therefore represents the spatio-temporal image. LetP=Diag(P, P, . . . P)represent an md by nd block-diagonal matrix obtained by replicating thesystem projection matrix P, d times. Now the spatio-temporal imagereconstruction problem may be formulated asY=Px+Ψwhere Y is the vector representing the spatio-temporal observations, Pis the block diagonal matrix representing the spatio-temporal projectionof the spatio-temporal image x onto the observations Y and Ψ is thespatio-temporal noise. This problem may be solved by finding aspatio-temporal image x that minimizes or maximizes an objectivefunction ƒ(Y, P, x, Ψ).

The image reconstruction unit 106 in the present embodiment isconfigured to solve the above-noted problem in another domain z insteadof in the spatio-temporal image domain x. The domain of z is selected ina way that the solution is likely to have a relatively small number ofnon-zero entries in z (i.e., the solution is sparse in the z domain).The domain of Y is the domain of spatio-temporal observations, alsoreferred to herein as the projection domain. The spatio-temporal domainsof x and z are referred to herein as the image domain and the sparsedomain, respectively.

The spatio-temporal domain of z is constructed using a transformationmatrix T, which may be obtained using one or more spatio-temporalmathematical transforms and the above-noted information from secondarysources. Formally,x=Tzwhere T is an nd by (n_(s)·n_(t)+n_(st)) spatio-temporal transformationmatrix. The transformation matrix T is constructed using n_(s) spatial,n_(t) temporal and n_(st) spatio-temporal basis functions. The crossproduct of the spatial and temporal basis functions gives n_(s)·n_(t)columns of the matrix T and the spatio-temporal basis functions give theremaining n_(st) columns of T.

Let b^(k)( ) represent the k^(th) spatio-temporal basis function. Wedenote the value of this function at a spatial location j and temporallocation t in the spatio-temporal domain as b^(k)(j, t) where k takesvalues from 1 to n_(st), j takes values from 1 to n and t takes valuesfrom 1 to d. The spatial basis functions are denoted as s¹(j) and thetemporal basis functions are denoted as τ^(k)(t) where l takes valuesfrom 1 to n_(s), j takes values from 1 to n, k takes values from 1 ton_(t), and t takes values from 1 to d.

The cross product of the spatial and temporal basis functions gives riseto n_(s)·n_(t) spatio-temporal basis functions as given below:c ^(lk)(j, t)=s ^(l)(j)·τ^(k)(t),where l takes values from 1 to n_(s), j takes values from 1 to n, ktakes values from 1 to n_(t) and t takes values from 1 to d.

It should be noted that the system matrix P and the transformationmatrix T are not required to be stored explicitly in the memory. Infact, in many state-of-the-art systems, matrices such as these will notbe explicitly stored but are instead implicitly computed on the fly. Thematrix-multiplication notations used herein may therefore be viewed asconvenient shorthand for the description of the corresponding algorithm.In practice, fast methods to compute the transforms (such as FFTs and/orwavelets) as well as fast methods to compute projection and backprojection without explicitly storing some portions of the matrices Tand P may be used, and such methods are well understood by those skilledin the art.

The transform domain basis functions can be obtained in several ways.For example, the basis functions can be constructed from spatial onlybasis functions multiplied with very simple temporal functions, temporalonly basis functions multiplied with very simple spatial functions,spatio-temporal basis functions and cross products of spatial andtemporal basis functions. There are at least two sources of the basisfunctions used in the construction. The first source comprises knownmathematical transforms used in signal processing, image processing,medical imaging, etc. The other source is secondary information.

As noted above, the secondary information may comprise specificinformation or general information, obtained from one or more secondarysources. The specific information may include information derived fromthe object being imaged, and the general information may includeinformation derived from either a mathematical model of the class of theobject being imaged or data obtained from multiple objects belonging tothe same class.

Examples of specific information about the object being imaged mayinclude a spatial, temporal or spatio-temporal image of the same objectobtained using another imaging modality. As a more particular example,specific information used to reconstruct a PET image may comprise an MRIand/or CT image of the same object. Another example of specificinformation is time series data representing radioactivity level inarterial blood of the person being imaged in a PET scanner. Such a timeseries may be obtained by periodically sampling the blood and measuringits radioactivity using another device. Similarly, for fMRI imagereconstruction, the specific information may be a high-resolution MRIimage, PET image or CT image of the same subject, or time series datarelating to heart beats, eye movement, respiration, head movements, orcharacteristics of the external stimulus administered to the subject(e.g., radioactivity concentration of the air being inhaled or someattributes of visual and/or auditory stimulus presented to the subjectwhile scanning is under progress), or other physical characteristics ofthe subject.

Examples of general information about the object being imaged mayinclude information derived from a physical or simulation-based model ofthe object or from multiple images of similar objects. As a moreparticular example, if the object being scanned is a human brain, thenthe general information may be different atlases of the human brainderived from scans of multiple subjects, or spatial or spatio-temporalbasis functions for specific parts of the brain based on singular valuedecomposition (SVD) of multiple brain scans.

The transform domain basis functions may more particularly be configuredas follows:

Spatial only basis functions multiplied with very simple temporalfunctions: In this construction, spatial basis functions such as 3D FFT,3D DCT, 3D spatial prior images, components of segmented spatial priorimages, etc. are crossed with very simple temporal functions such asshifted delta functions to form the basis.

Temporal only basis functions multiplied with very simple spatialfunctions: In this construction, temporal basis functions such asB-splines or temporal signals derived using information from secondarysources are crossed with very simple spatial basis function such aspixel basis functions.

Spatio-temporal basis functions: These may include 4D mathematicaltransforms such as 4D FFT, 4D DCT, 4D wavelet transform, 4D HAARtransform, and prior spatio-temporal images.

Cross product of spatial and temporal basis functions: In thisconstruction, spatial basis functions are crossed with temporal basisfunctions. The spatial basis functions may include 3D mathematicaltransforms such as 3D FFT, 3D DCT, 3D HAAR, Curvelet transform, prior 3Dspatial images, segmented image components from priors, generalinformation based basis like brain atlases using multiple brain scans,heart models derived from physics and other heart scans, SVD dataderived from multiple brain scans, etc. The temporal basis functions mayinclude mathematical transforms such as 1D Wavelet transform, 1DB-splines, 1D FFT, and specific secondary information basis functionssuch as heart rate, heart beat and pulse breathing information.

As mentioned previously, the image reconstruction unit 106 is configuredto determine a spatio-temporal image x that minimizes or maximizes anobjective function ƒ(Y, P, x, Ψ), but solves this problem in the sparsedomain z instead of in the spatio-temporal domain x. Although manydifferent types of objective functions may be used, two particularexamples based on log-likelihood and least square functions are shownbelow:

1. Log-likelihood function:

${f\left( {Y,\underset{\_}{P},T,z,\psi} \right)} = {{\sum\limits_{i = 0}^{i = {{md} - 1}}\;{Y_{i}{\log\left( {{\underset{\_}{P}{Tz}} - \psi} \right)}_{i}}} - \left( {{\underset{\_}{P}{Tz}} - \psi} \right)_{i} - {\log\; Y_{i}} + {R(x)}}$

2. Least square function:ƒ(Y, P, T, z, ψ)=∥Y−PTz−ψ∥ ² −R(x)

where:

Y is the stacked observation

T is the transform matrix

P is the block diagonal system matrix

z is the spatio-temporal image in the sparse domain

x=Tz is the spatio-temporal image

Ψ is the spatio-temporal noise

R(x) is a penalty term, also referred to as the regularization penalty

The penalty term is added or subtracted depending on the minimization ormaximization goal of the objective function in order to obtain smootherand less noisy reconstruction output. The penalty term may take any of anumber of different forms. For example, the penalty term may be one ofthe norms of the image, e.g., R(x^(t))=(Σ_(i=1) ^(n)|x_(i)|^(q))^(1/q),where q represents the q^(th) norm, or it can also consider pixelneighbors of the pixel, e.g.,

${R\left( x^{t} \right)} = \left( {\sum\limits_{i = 1}^{n}\;{\sum\limits_{k \in N_{i}}\;{w_{ik}{\varphi\left( {x_{i},x_{k}} \right)}}}} \right)$where φ captures the penalty to be incurred depending on the values ofthe neighboring pixels in the image. The actual choice of penalty termcan be guided by the previously-described secondary information aboutthe object being imaged. For example, if it is known that thereconstruction output should be smooth, then the second penalty termabove may be more appropriate. However, it may create false artifactsacross the boundaries of different segments of the image. To avoid this,the selection of penalty term may further be guided by secondaryinformation based on segmented image components.

The spatio-temporal reconstruction problem described above can be solvedusing known optimization techniques such as, for example, gradientdescent, conjugate gradient descent, or preconditioned conjugategradient descent, using one of the objective functions.

Referring now to FIG. 3, a flow diagram of the spatio-temporal imagereconstruction process implemented in image reconstruction unit 106 isshown. The process in this particular embodiment includes Steps 1through 10, each of which will be described below.

Step 1. The image domain representation is initialized, for example, toconstant value, or by running an analytical reconstruction algorithmsuch as OSEM3D, which is described in I. K. Hong et al., “Ultra fastsymmetry and simd-based projection-backprojection (ssp) algorithm for3-d pet image reconstruction,” IEEE Trans. Med. Imaging, 26(6):789-803,2007. Although not explicitly indicated in this step of the figure, theinitialized image may then be transformed using the transform T toobtain an initial transform domain representation.

Step 2. The projections Y^(i) are computed using image domainrepresentation x and system matrix P.

Step 3. A residue is computed, using projected values and experimentalobservation, in order to quantify the quality of the current solution.This may involve, for example, checking the value of one of the norms(e.g., L2 norm) of (Y−Y^(i)−Ψ), checking if the desired number ofiterations has been performed, etc.

Step 4. A determination is made as to whether or not the residuecomputed in Step 3 indicates that the current solution is of sufficientquality.

Step 5. The process ends if the current solution is of sufficientquality, and otherwise proceeds to Step 6.

Step 6. The desired direction of improvement in the projection domain iscomputed using any iterative optimization technique as applied to theobjective function. As indicated above, one example of a suitableoptimization technique is gradient descent. These and other knownoptimization techniques are described in, for example, Jan A. Snyman,“Practical Mathematical Optimization: An Introduction to BasicOptimization Theory and Classical and New Gradient-Based Algorithms,”Springer Publishing (2005), ISBN 0-387-24348-8.

Step 7. The projection domain desired direction of improvement is backprojected using P to get the desired direction of improvement in theimage domain representation x. As indicated previously, forward and backprojection methods are known in the art, and therefore will not bedescribed in detail herein.

Step 8. The image domain desired direction of improvement is transformedinto the sparse domain desired direction of improvement using T.

Step 9. A sparse solution is found in the sparse domain using thedesired direction of improvement and the sparse domain representation z.

Step 10. An updated image x is obtained from the sparse domain solutionusing T and the sparse and image domain representations are updated tothe new solution. The process then returns to Step 2 as shown, andcontinues in an iterative manner until the solution converges to anacceptable result.

The particular image reconstruction process shown in FIG. 3 is presentedby way of illustrative example only, and other embodiments may utilizeother arrangements of process steps to reconstruct a spatio-temporalimage.

As will be appreciated by one skilled in the art, aspects of the presentinvention may be embodied as a system, method or computer programproduct. Accordingly, aspects of the present invention may take the formof an entirely hardware embodiment, an entirely software embodiment(including firmware, resident software, micro-code, etc.) or anembodiment combining software and hardware aspects that may allgenerally be referred to herein as a “circuit,” “module” or “system.”Furthermore, aspects of the present invention may take the form of acomputer program product embodied in one or more computer readablemedium(s) having computer readable program code embodied thereon. Agiven computer readable medium of this type may be part of or otherwiseassociated with a processor such as the above-noted GPUs.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk,RAM, ROM, an erasable programmable read-only memory (EPROM or Flashmemory), an optical fiber, a portable compact disc read-only memory(CD-ROM), an optical storage device, a magnetic storage device, or anysuitable combination of the foregoing. In the context of this document,a computer readable storage medium may be any tangible medium that cancontain, or store a program for use by or in connection with aninstruction execution system, apparatus, or device.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wireline, optical fiber cable, RF, etc., or any suitable combination ofthe foregoing.

Computer program code for carrying out operations for aspects of thepresent invention may be written in any combination of one or moreprogramming languages, including an object oriented programming languagesuch as Java, Smalltalk, C++ or the like and conventional proceduralprogramming languages, such as the “C” programming language or similarprogramming languages. The program code may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider). A given computer or server may comprise one or moreprocessors, such as the above-noted GPUs.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

The flowchart and block diagrams in the figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

It should therefore again be emphasized that the various embodimentsdescribed herein are presented by way of illustrative example only, andshould not be construed as limiting the scope of the invention. Forexample, alternative embodiments of the invention can utilize differentimage processing system and image reconstruction unit configurations,and different reconstruction processes, than those described above inthe context of the illustrative embodiments. Also, although particularlywell suited for use in medical imaging applications, the disclosedtechniques can be adapted in a straightforward manner for use in a widevariety of other imaging applications, including 2D or 3D moviesinvolving people and/or models. These and numerous other alternativeembodiments within the scope of the appended claims will be readilyapparent to those skilled in the art.

What is claimed is:
 1. A method of generating an image, the methodcomprising: obtaining captured data characterizing an object;reconstructing a spatio-temporal image of the object based on thecaptured data, the spatio-temporal image comprising a plurality ofspatial images in respective time intervals, with at least a given oneof the spatial images in one of the time intervals being reconstructedusing captured data from a frame associated with that time interval andcaptured data associated with one or more additional frames associatedwith other time intervals; and outputting the spatio-temporal image;wherein the obtaining, reconstructing and outputting steps are performedby a processing device comprising a processor coupled to a memory;wherein the step of reconstructing a spatio-temporal image furthercomprises obtaining a solution to a minimization or maximization problemin a sparse domain and transforming the solution to an image domain; andwherein the maximization or minimization problem is based on one of alog-likelihood objective function and a least square objective function.2. The method of claim 1 wherein the log-likelihood objective functionis given by:${f\left( {Y,\underset{\_}{P},T,z,\psi} \right)} = {{\sum\limits_{i = 0}^{i = {{md} - 1}}\;{Y_{i}{\log\left( {{\underset{\_}{P}{Tz}} - \psi} \right)}_{i}}} - \left( {{\underset{\_}{P}{Tz}} - \psi} \right)_{i} - {\log\; Y_{i}} + {R(x)}}$where Y is a projection domain representation, T is a transformationmatrix between the image domain and the sparse domain, P is a blockdiagonal system matrix, z is a sparse domain representation, x=Tz is animage domain representation, Ψ represents spatio-temporal noise, andR(x) is a penalty term.
 3. The method of claim 1 wherein the leastsquare objective function is given by:ƒ(Y, P, T, z, ψ)=∥Y−PTz−ψ∥ ² −R(x) where Y is a projection domainrepresentation, T is a transformation matrix between the image domainand the sparse domain, P is a block diagonal system matrix, z is asparse domain representation, x=Tz is an image domain representation, Ψrepresents spatio-temporal noise, and R(x) is a penalty term.
 4. Acomputer program product comprising a non-transitory computer readablestorage medium having computer readable program code embodied therewith,the computer readable program code comprising computer readable programcode configured to perform the steps of the method of claim
 1. 5. Anapparatus comprising: an image reconstruction unit; the imagereconstruction unit comprising a processor; wherein the imagereconstruction unit is operative under control of the processor toobtain captured data characterizing an object, and to reconstruct aspatio-temporal image of the object based on the captured data; whereinthe spatio-temporal image comprises a plurality of spatial images inrespective time intervals and at least a given one of the spatial imagesin one of the time intervals is reconstructed using captured data from aframe associated with that time interval and captured data associatedwith one or more additional frames associated with other time intervals;wherein the image reconstruction unit is operative to reconstruct thespatio-temporal image by obtaining a solution to a minimization ormaximization problem in a sparse domain and transforming the solution toan image domain; and wherein the maximization or minimization problem isbased on one of a log-likelihood objective function and a least squareobjective function.
 6. The apparatus of claim 5 wherein the imagereconstruction unit comprises a sparse regression module and secondaryinformation module.
 7. The apparatus of claim 5 wherein the imagereconstruction unit is implemented in a computer.
 8. An integratedcircuit comprising the apparatus of claim
 5. 9. The apparatus of claim 5wherein the log-likelihood objective function is given by:${f\left( {Y,\underset{\_}{P},T,z,\psi} \right)} = {{\sum\limits_{i = 0}^{i = {{md} - 1}}{Y_{i}{\log\left( {{\underset{\_}{P}{Tz}} - \psi} \right)}_{i}}} - \left( {{\underset{\_}{P}{Tz}} - \psi} \right)_{i} - {\log\; Y_{i}} + {R(x)}}$where Y is a projection domain representation, T is a transformationmatrix between the image domain and the sparse domain, P is a blockdiagonal system matrix, z is a sparse domain representation, x=Tz is animage domain representation, Ψ represents spatio-temporal noise, andR(x) is a penalty term.
 10. The apparatus of claim 5 wherein the leastsquare objective function is given by:ƒ(Y, P, T, z, ψ)=∥Y−PTz−ψ∥ ² −R(x) where Y is a projection domainrepresentation, T is a transformation matrix between the image domainand the sparse domain, P is a block diagonal system matrix, z is asparse domain representation, x=Tz is an image domain representation, Ψrepresents spatio-temporal noise, and R(x) is a penalty term.
 11. Theapparatus of claim 5 wherein the captured data comprises data capturedfrom a medical image scanner.
 12. The apparatus of claim 11 wherein themedical image scanner comprises one of magnetic-resonance imagingscanner, a positron emission tomography scanner and a computedtomography scanner.
 13. The apparatus of claim 6 wherein the imagereconstruction unit is configured to utilize information received fromthe secondary information module to reconstruct the spatio-temporalimage.
 14. The apparatus of claim 13 wherein the information receivedfrom the secondary information module comprises information derived fromthe object being imaged.
 15. The apparatus of claim 13 wherein theinformation received from the secondary information module comprisesinformation derived from one of a mathematical model of a class of theobject being imaged and data obtained from multiple objects in theclass.
 16. The method of claim 1 wherein the captured data comprisesdata captured from a medical image scanner.
 17. The method of claim 16wherein the medical image scanner comprises one of magnetic-resonanceimaging scanner, a positron emission tomography scanner and a computedtomography scanner.
 18. The method of claim 1 wherein reconstructing thespatio-temporal image is also based on secondary information.
 19. Themethod of claim 18 wherein the secondary information comprisesinformation derived from the object being imaged.
 20. The method ofclaim 18 wherein the secondary information comprises information derivedfrom one of a mathematical model of a class of the object being imagedand data obtained from multiple objects in the class.